Correction of unfocus and misfocus via origin-centered discrete fractional fourier transform

ABSTRACT

A method corrects unfocus and misfocus in original image data using an origin-centered two-dimensional discrete fractional Fourier transform. The origin-centered two-dimensional discrete fractional Fourier transform numerically corrects a modeled evolution of an image propagating through a medium, aligning a propagation model centerline with the center of image data and the center an index-shifted numerical fractional Fourier transform operation. A zero transform-image domain origin is accordingly aligned to the center of modified image data. The corrective fractional Fourier transform power may be determined automatically or by human operator. The image correction can be applied to lens-systems or other systems obeying Fourier optics including electron microscopes, particle beams, radiation accelerators. The invention can be incorporated into cameras, film processing, photo editing software, websites, video recorders, video editing systems, surveillance systems, conferencing systems, image processing algorithms, optical computing systems, and other types of system, products and service facilities.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.13/037,342, filed Feb. 28, 2011, which is a continuation of U.S. Reissueapplication Ser. No. 12/101,878, filed Apr. 11, 2008, for U.S. Pat. No.7,039,252 issued on May 2, 2006 (U.S. application Ser. No. 10/980,744filed on Nov. 2, 2004, which is a continuation-in-part of applicationSer. No. 10/665,439, filed on Sep. 18, 2003, which is acontinuation-in-part of application Ser. No. 09/512,775, filed on Feb.25, 2000, now U.S. Pat. No. 6,687,418 claiming priority to provisionalapplication No. 60/121,680 and to provisional application No.60/121,958, both filed on Feb. 25, 1999) (i.e., a Bauman typecontinuation application), now U.S. Pat. No. RE 42,187.

BACKGROUND OF THE INVENTION

1. Field of the Invention

This invention relates to optical signal processing, and moreparticularly to the use of fractional Fourier transform properties oflenses to correct the effects of lens misfocus in photographs, video,and other types of captured images.

2. Discussion of the Related Art

A number of references are cited herein; these are provided in anumbered list at the end of the detailed description of the preferredembodiments. These references are cited at various locations throughoutthe specification using a reference number enclosed in square brackets.

The Fourier transforming properties of simple lenses and related opticalelements is well known and heavily used in a branch of engineering knownas Fourier optics [1, 2]. Classical Fourier optics [1, 2, 3, 4] utilizelenses or other means to obtain a two-dimensional Fourier transform ofan optical wavefront, thus creating a Fourier plane at a particularspatial location relative to an associated lens. This Fourier planeincludes an amplitude distribution of an original two-dimensionaloptical image, which becomes the two-dimensional Fourier transform ofitself. In the far simpler area of classical geometric optics [1, 3],lenses and related objects are used to change the magnification of atwo-dimensional image according to the geometric relationship of theclassical lens-law. It has been shown that between the geometriesrequired for classical Fourier optics and classical geometric optics,the action of a lens or related object acts on the amplitudedistribution of images as the fractional power of the two-dimensionalFourier transform. The fractional power of the fractional Fouriertransform is determined by the focal length characteristics of the lens,and the relative spatial separation between a lens, source image, and anobserved image.

The fractional Fourier transform has been independently discovered onvarious occasions over the years [5, 7, 8, 9, 10], and is related toseveral types of mathematical objects such as the Bargmann transform [8]and the Hermite semigroup [13]. As shown in [5], for example, the mostgeneral form of optical properties of lenses and other related elements[1, 2, 3] can be transformed into a fractional Fourier transformrepresentation. This property has apparently been rediscovered someyears later and worked on steadily ever since (see for example [6]),expanding the number of optical elements and situations covered. It isimportant to remark, however, that the lens modeling approach in thelatter ongoing series of papers view the multiplicative phase term inthe true form of the fractional Fourier transform as a problem orannoyance and usually omit it from consideration.

SUMMARY OF THE INVENTION

Correction of the effects of misfocusing in recorded or real-time imagedata may be accomplished using fractional Fourier transform operationsrealized optically, computationally, or electronically. In someembodiments, the invention extends the capabilities of using a power ofthe fractional Fourier transform for correcting misfocused images, tosituations where phase information associated with the original imagemisfocus is unavailable. For example, conventional photographic andelectronic image capture, storage, and production technologies can onlycapture and process image amplitude information—the relative phaseinformation created within the original optical path is lost. As will bedescribed herein, the missing phase information can be reconstructed andused when correcting image misfocus.

In accordance with embodiments of the invention, a method forapproximating the evolution of images propagating through a physicalmedium is provided by calculating a fractional power of a numericaloperator. The numerical operator may be defined by the physical mediumand includes a diagonalizable numerical linear operator raised to apower (α). The method further includes representing a plurality ofimages using an individual data array for each of the images. Thenumerical operator may be represented with a linear operator formed bymultiplying an ordered similarity transformation operator (P) by acorrespondingly-ordered diagonal operator (Λ), the result of which ismultiplied by an approximate inverse (P⁻¹) of the ordered similaritytransformation operator (P). Diagonal elements of thecorrespondingly-ordered diagonal operator (Λ) may be raised to the power(α) to produce a fractional power diagonal operator. The fractionalpower diagonal operator may be multiplied by an approximate inverse ofthe ordered similarity transformation operator (P⁻¹) to produce a firstpartial result. In addition, the data array of one of the images may bemultiplied by the ordered similarity transformation operator (P) toproduce a modified data array. The modified data array may then bemultiplied by the first partial result to produce the fractional powerof the numerical operator. If desired, the last-two multiplicationoperations may be repeated for each of a plurality of images.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other aspects, features and advantages of the presentinvention will become more apparent upon consideration of the followingdescription of preferred embodiments taken in conjunction with theaccompanying drawing figures, wherein:

FIG. 1 is a block diagram showing a general lens arrangement andassociated image observation entity capable of classical geometricoptics, classical Fourier optics, and fractional Fourier transformoptics;

FIG. 2 is a block diagram showing an exemplary approach for automatedadjustment of fractional Fourier transform parameters for maximizing thesharp edge content of a corrected image, in accordance with oneembodiment of the present invention;

FIG. 3 is a block diagram showing a typical approach for adjusting thefractional Fourier transform parameters to maximize misfocus correctionof an image, in accordance with one embodiment of the present invention;

FIG. 4 is a diagram showing a generalized optical environment forimplementing image correction in accordance with the present invention;

FIG. 5 is a diagram showing focused and unfocused image planes inrelationship to the optical environment depicted in FIG. 4;

FIG. 6 is a block diagram showing an exemplary image misfocus correctionprocess that also provides phase corrections;

FIG. 7 is a diagram showing a more detailed view of the focused andunfocused image planes shown in FIG. 5;

FIG. 8 is a diagram showing typical phase shifts involved in the focusedand unfocused image planes depicted in FIG. 5;

FIG. 9 shows techniques for computing phase correction determined by thefractional Fourier transform applied to a misfocused image;

FIG. 10 is a block diagram showing an exemplary image misfocuscorrection process that also provides for phase correction, inaccordance with an alternative embodiment of the invention;

FIG. 11 shows a diagonalizable matrix, tensor, or linear operator actingon an underlying vector, matrix, tensor, or function;

FIG. 12 is a flowchart showing exemplary operations for approximatingthe evolution of images propagating through a physical medium, inaccordance with embodiments of the invention;

FIG. 13 is a flowchart showing exemplary operations for approximatingthe evolution of images propagating through a physical medium, inaccordance with alternative embodiments of the invention.

FIG. 14 shows a simplified version of the equation depicted in Figure

FIG. 15 shows an exemplary matrix;

FIGS. 16A through 16C are side views of a propagating light or particlebeam;

FIG. 16D is a top view of contours of constant radial displacement withrespect to the image center of paths associated with the imagepropagation of an image;

FIG. 17 shows an exemplary matrix for a centered normalized classicaldiscrete Fourier transform;

FIG. 18A is a graph showing a frequency-domain output of a non-centeredclassical discrete Fourier transform for an exemplary signal;

FIG. 18B is a graph showing a frequency-domain output of a centeredclassical discrete Fourier transform for the same exemplary signal;

FIG. 19 shows the coordinate indexing of an image having a particularheight and width; and

FIG. 20 is a block diagram showing the isolation and later reassembly ofquadrant portions of an original image.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

In the following description, reference is made to the accompanyingdrawing figures which form a part hereof, and which show by way ofillustration specific embodiments of the invention. It is to beunderstood by those of ordinary skill in this technological field thatother embodiments may be utilized, and structural, electrical, optical,as well as procedural changes may be made without departing from thescope of the present invention.

As used herein, the term “image” refers to both still-images (such asphotographs, video frames, video stills, movie frames, and the like) andmoving images (such as motion video and movies). Many embodiments of thepresent invention are directed to processing recorded or real-time imagedata provided by an exogenous system, means, or method. Presented imagedata may be obtained from a suitable electronic display such as an LCDpanel, CRT, LED array, films, slides, illuminated photographs, and thelike. Alternatively or additionally, the presented image data may be theoutput of some exogenous system such as an optical computer orintegrated optics device, to name a few. The presented image data willalso be referred to herein as the image source.

If desired, the system may output generated image data having someamount of misfocus correction. Generated image data may be presented toa person, sensor (such as a CCD image sensor, photo-transistor array,for example), or some exogenous system such as an optical computer,integrated optics device, and the like. The entity receiving generatedimage data will be referred to as an observer, image observation entity,or observation entity.

Reference will first be made to FIG. 3 which shows a general approachfor adjusting the fractional Fourier transform parameters to maximizethe correction of misfocus in an image. Details regarding the use of afractional Fourier transform (with adjusted parameters of exponentialpower and scale) to correct image misfocus will be later described withregard to FIGS. 1 and 2.

Original visual scene 301 (or other image source) may be observed byoptical system 302 (such as a camera and lens arrangement) to produceoriginal image data 303 a. In accordance with some embodiments, opticalsystem 302 may be limited, misadjusted, or otherwise defective to theextent that it introduces a degree of misfocus into the imagerepresented by the image data 303 a. It is typically not possible orpractical to correct this misfocus effect at optical system 302 toproduce a better focused version of original image data 303 a.Misfocused original image data 303 a may be stored over time ortransported over distance. During such a process, the original imagedata may be transmitted, converted, compressed, decompressed, orotherwise degraded, resulting in an identical or perturbed version oforiginal image data 303 b. It is this perturbed version of the originalimage data that may be improved using the misfocus correction techniquesdisclosed herein. Original and perturbed image data 303 a, 303 b may bein the form of an electronic signal, data file, photography paper, orother image form.

Original image data 303 b may be manipulated numerically, optically, orby other means to perform a fractional Fourier transform operation 304on the original image data to produce resulting (modified) image data305. The parameters of exponential power and scale factors of thefractional Fourier transform operation 304 may be adjusted 310 over somerange of values, and each parameter setting within this range may resultin a different version of resulting image data 305. As the level ofmisfocus correction progresses, the resulting image data 305 will appearmore in focus. The improvement in focus will generally be obvious to anattentive human visual observer, and will typically be signified by anincrease in image sharpness, particularly at any edges that appear inthe image. Thus a human operator, a machine control system, or acombination of each can compare a sequence of resulting images createdby previously selected parameter settings 310, and try a new parametersetting for a yet another potential improvement.

For a human operator, this typically would be a matter of adjusting acontrol and comparing images side by side (facilitated by non-humanmemory) or, as in the case of a microscope or telescope, by comparisonfacilitated purely with human memory. For a machine, a systematiciterative or other feedback control scheme would typically be used.

In FIG. 3, each of these image adjustments is generalized by the stepsand elements suggested by interconnected elements 306-309, althoughother systems or methods accomplishing the same goal with differentinternal structure (for example, an analog electronic circuit, opticalmaterials, or chemical process) are provided for and anticipated by thepresent invention. For the illustrative general case of FIG. 3,resulting image data 305 for selected parameter settings 310 may bestored in human, machine, or photographic memory 306, along with theassociated parameter settings, and compared 307 for the quality of imagefocus. Based on these comparisons, subsequent high level actions 308 maybe chosen.

High level actions 308 typically require translation into new parametervalues and their realization, which may be provided by parametercalculation and control 309. This process may continue for some intervalof time, some number of resulting images 305, or some chosen orpre-determined maximum level of improvement. One or more “best choice”resulting image data set or sets 305 may then be identified as theresult of the action and processes depicted in this figure.

With this high level description having been established, attention isnow directed to details of the properties and use of a fractionalFourier transform (with adjusted parameters of exponential power andscale) to correct misfocus in an image and maximize correction ofmisfocus. This aspect of the present invention will be described withregard to FIG. 1.

FIG. 1 is a block diagram showing image source 101, lens 102, and imageobservation entity 103. The term “lens” is used herein for convenience,but it is to be understood that the image misfocus correction techniquesdisclosed herein apply equally to lens systems and other similar opticalenvironments. The image observation entity may be configured withclassical geometric optics, classical Fourier optics, or fractionalFourier transform optics. The particular class of optics (geometric,Fourier, or fractional Fourier) implemented in a certain application maybe determined using any of the following:

-   -   separation distances 111 and 112;    -   the “focal length” parameter “f” of lens 102;    -   the type of image source (lit object, projection screen, etc.)        in as far as whether a plane or spherical wave is emitted.

As is well known, in situations where the source image is a lit objectand where distance 111, which shall be called “a,” and distance 112,which shall be called “b,” fall into the lens-law relationship, may bedetermined by the focal length f:

$\begin{matrix}{\frac{1}{f} = {\frac{1}{a} + \frac{1}{b}}} & (1)\end{matrix}$which gives the geometric optics case. In this case, observed image 103is a vertically and horizontally inverted version of the original imagefrom source 101, scaled in size by a magnification factor m given by:

$\begin{matrix}{m = \frac{b}{a}} & (2)\end{matrix}$

As previously noted, the Fourier transforming properties of simplelenses and related optical elements is also well known in the field ofFourier optics [2,3]. Classical Fourier optics [2, 3, 4, 5] involve theuse of a lens, for example, to take a first two-dimensional Fouriertransform of an optical wavefront, thus creating a Fourier plane at aparticular spatial location such that the amplitude distribution of anoriginal two-dimensional optical image becomes the two-dimensionalFourier transform of itself. In the arrangement depicted in FIG. 1, witha lit object serving as source image 101, the Fourier optics case may beobtained when a=b=f.

As described in [5], for cases where a, b, and f do not satisfy the lenslaw of the Fourier optics condition above, the amplitude distribution ofsource image 101, as observed at observation entity 103, experiences theaction of a non-integer power of the Fourier transform operator. Asdescribed in [5], this power, which shall be called α, varies between 0and 2 and is determined by an Arc-Cosine function dependent on the lensfocal length and the distances between the lens, image source, and imageobserver; specifically:

$\begin{matrix}{\alpha = {\frac{2}{\pi}{\arccos\lbrack {{{sgn}( {f - a} )}\frac{\sqrt{( {f - a} )( {f - b} )}}{f}} \rbrack}}} & (3)\end{matrix}$for cases where (f−a) and (f−b) share the same sign. There are othercases which can be solved from the more primitive equations in [5] (atthe bottom of pages ThE4-3 and ThE4-1). Note simple substitutions showthat the lens law relationship among a, b, and f indeed give a power of2, and that the Fourier optics condition of a=b=f give the power of 1,as required.

The fractional Fourier transform properties of lenses typically causecomplex but predictable phase and scale variations. These variations maybe expressed in terms of Hermite functions, as presented shortly, but itis understood that other representations of the effects, such asclosed-form integral representations given in [5], are also possible anduseful.

Various methods can be used to construct the fractional Fouriertransform, but to begin it is illustrative to use the orthogonal Hermitefunctions, which as eigenfunctions diagonalize the Fourier transform[17]. Consider the Hermite function [16] expansion [17, and morerecently, 18] of the two dimensional image amplitude distributionfunction. In one dimension, a bounded (i.e., non-infinite) function k(x)can be represented as an infinite sum of Hermite functions {h_(n)(x)}as:

$\begin{matrix}{{k(x)} = {\sum\limits_{n = 0}^{\infty}\;{a_{n}{h_{n}(x)}}}} & (4)\end{matrix}$

Since the function is bounded, the coefficients {a_(n)} eventuallybecome smaller and converge to zero. An image may be treated as a twodimensional entity (for example, a two-dimensional array of pixels), orit can be the amplitude variation of a translucent plate; in eithercase, the function may be represented in a two-dimensional expansionsuch as:

$\begin{matrix}{{k( {x_{1},x_{2}} )} = {\sum\limits_{m = 0}^{\infty}{\sum\limits_{n = 0}^{\infty}{a_{n,m}{h_{n}( x_{1} )}{h_{m}( x_{2} )}}}}} & (5)\end{matrix}$For simplicity, the one dimensional case may be considered. The Fouriertransform action on Hermite expansion of the function k(x) with seriescoefficients {a_(n)} is given by [16]:

$\begin{matrix}{{F\lbrack {k(x)} \rbrack} = {\sum\limits_{n = 0}^{\infty}{( {- {\mathbb{i}}} )^{n}a_{n}{h_{n}(x)}}}} & (6)\end{matrix}$Because of the diagonal eigenfunction structure, fractional powers ofthe Fourier transform operator may be obtained by taking the fractionalpower of each eigenfunction coefficient. The eigenfunction coefficientshere are (−i)^(n). Complex branching artifact ambiguities that arisefrom taking the roots of complex numbers can be avoided by writing (−i)as:e ^(−iπ/2)  (7)Thus for a given power α, the fractional Fourier transform of theHermite expansion of the function k(x) with series coefficients{a_(n)}can be given by [5]:

$\begin{matrix}{{F^{\alpha}\lbrack {k(x)} \rbrack} = {\sum\limits_{n = 0}^{\infty}{{\mathbb{e}}^{{- {\mathbb{i}}}\; n\;{{\pi\alpha}/2}}a_{n}{h_{n}(x)}}}} & (8)\end{matrix}$Note when α=1, the result is the traditional Fourier transform above,and when α=2, the result may be expressed as:

$\begin{matrix}\begin{matrix}{{F^{2}\lbrack {k(x)} \rbrack} = {\sum\limits_{n = 0}^{\infty}{{\mathbb{e}}^{{- {\mathbb{i}}}\; n\;\pi}a_{n}{h_{n}(x)}}}} \\{= {\sum\limits_{n = 0}^{\infty}{( {- 1} )^{n}a_{n}{h_{n}(x)}}}} \\{= {\sum\limits_{n = 0}^{\infty}{a_{n}{h_{n}( {- x} )}}}} \\{= {k( {- x} )}}\end{matrix} & (9)\end{matrix}$due to the odd and even symmetry, respectively, of the odd and evenHermite functions. This is the case for the horizontally and verticallyinverted image associated with the lens law of geometric optics,although here the scale factors determining the magnification factorhave been normalized out.

More generally, as the power α varies (via the Arccosine relationshipdepending on the separation distance), the phase angle of the n^(th)coefficient of the Hermite expansion varies according to therelationship shown above and the scale factor may vary as well [5]. Forimages, all of the above occurs in the same manner but in two dimensions[5].

Through use of the Mehler kernel [16], the above expansion may berepresented in closed form as [5]:

$\begin{matrix}{{F^{\alpha}\lbrack {k(x)} \rbrack} = {\sqrt{\frac{{\mathbb{e}}^{{- {\pi\alpha\mathbb{i}}}/2}}{{\mathbb{i}}\;{\sin( {{\pi\alpha}/2} )}}}{\int_{- \infty}^{\infty}{{k(x)}{\mathbb{e}}^{2{{\pi\mathbb{i}}\lbrack{{{(\frac{x^{2} + y^{2}}{2})}{\cot{(\frac{\pi\alpha}{2})}}} - {{xycsc}{(\frac{\pi\alpha}{2})}}}\rbrack}}{\mathbb{d}x}}}}} & (10)\end{matrix}$Note in [5] that the factor of i multiplying the sin function under theradical has been erroneously omitted. Clearly, both the Hermite andintegral representations are periodic in α with period four. Further, itcan be seen from either representation that:F ^(2±ε) [k(x)]=F ² F ^(±ε) [k(x)]=F ^(±ε) F ² [k(x)]=F ^(±ε)[k(−x)]  (11)which illustrates an aspect of the invention as the effect ε will be thedegree of misfocus introduced by the misfocused lens, while the Fouriertransform raised to the second power represents the lens-law opticscase. In particular, the group property makes it possible to calculatethe inverse operation to the effect induced on a record image by amisfocused lens in terms of explicit mathematical operations that can berealized either computationally, by means of an optical system, or both.Specifically, because the group has period 4, it follows that F⁻²=F²;thus:(F ^(2±ε) [k(x)])⁻¹ =F ⁻² F ^(∓ε) [k(x)]=F ² F ^(∓ε) [k(x)]=F ^(∓ε) F ²[k(x)]=F ^(∓ε) [k(−x)]  (12)

Thus, one aspect of the invention provides image misfocus correction,where the misfocused image had been created by a quality thoughmisfocused lens or lens-system. This misfocus can be corrected byapplying a fractional Fourier transform operation; and morespecifically, if the lens is misfocused by an amount corresponding tothe fractional Fourier transform of power ε, the misfocus may becorrected by applying a fractional Fourier transform operation of power−ε.

It is understood that in some types of situations, spatial scale factorsof the image may need to be adjusted in conjunction with the fractionalFourier transform power. For small variations of the fractional Fouriertransform power associated with a slight misfocus, this is unlikely tobe necessary. However, should spatial scaling need to be made, variousoptical and signal processing methods well known to those skilled in theart can be incorporated. In the case of pixilated images (imagesgenerated by digital cameras, for example) or lined-images (generated byvideo-based systems, for example), numerical signal processingoperations may require standard resampling (interpolation and/ordecimation) as is well known to those familiar with standard signalprocessing techniques.

It is likely that the value of power ε is unknown a priori. In thisparticular circumstance, the power of the correcting fractional Fouriertransform operation may be varied until the resulting image is optimallysharpened. This variation could be done by human interaction, as withconventional human interaction of lens focus adjustments on a camera ormicroscope, for example.

If desired, this variation could be automated using, for example, somesort of detector in an overall negative feedback situation. Inparticular, it is noted that a function with sharp edges are obtainedonly when its contributing, smoothly-shaped basis functions have veryparticular phase adjustments, and perturbations of these phaserelationships rapidly smooth and disperse the sharpness of the edges.Most natural images contain some non-zero content of sharp edges, andfurther it would be quite unlikely that a naturally occurring, smoothgradient would tighten into a sharp edge under the action of thefractional Fourier transform because of the extraordinary basis phaserelationships required. This suggests that a spatial high-pass filter,differentiator, or other edge detector could be used as part of thesensor makeup. In particular, an automatically adjusting system may beconfigured to adjust the fractional Fourier transform power to maximizethe sharp edge content of the resulting correcting image. If desired,such a system may also be configured with human override capabilities tofacilitate pathological image situations, for example.

FIG. 2 shows an automated approach for adjusting the fractional Fouriertransform parameters of exponential power and scale factor to maximizethe sharp edge content of the resulting correcting image. In thisfigure, original image data 201 is presented to an adjustable fractionalFourier transform element 202, which may be realized physically viaoptical processes or numerically (using an image processing orcomputation system, for example). The power and scale factors of thefractional Fourier transform may be set and adjusted 203 as necessaryunder the control of a step direction and size control element 204.

Typically, this element would initially set the power to the ideal valueof zero (making the resulting image data 205 equivalent to the originalimage data 201) or two (making the resulting image data 205 equivalentto an inverted image of original image data 201), and then deviateslightly in either direction from this initial value. The resultingimage data 205 may be presented to edge detector 206 which identifiesedges, via differentiation or other means, whose sharpness passes aspecified fixed or adaptive threshold. The identified edge informationmay be passed to an edge percentage tally element 207, which transformsthis information into a scalar-valued measure of the relative degree ofthe amount of edges, using this as a measure of image sharpness.

The scalar measure value for each fractional Fourier transform power maybe stored in memory 208, and presented to step direction and sizecontrol element 204. The step direction and size control elementcompares this value with the information stored in memory 208 andadjusts the choice of the next value of fractional Fourier transformpower accordingly. In some implementations, the step direction and sizecontrol element may also control edge detection parameters, such as thesharpness threshold of edge detector element 207. When the optimaladjustment is determined, image data 205 associated with the optimalfractional Fourier transform power is designated as the corrected image.

It is understood that the above system amounts to a negative-feedbackcontrol or adaptive control system with a fixed or adaptive observer. Assuch, it is understood that alternate means of realizing this automatedadjustment can be applied by those skilled in the art. It is also clearto one skilled in the art that various means of interactive humanintervention may be introduced into this automatic system to handleproblem cases or as a full replacement for the automated system.

In general, the corrective fractional Fourier transform operation can beaccomplished by any one or combination of optical, numerical computer,or digital signal processing methods as known to those familiar with theart, recognizing yet other methods may also be possible. Optical methodsmay give effectively exact implementations of the fractional Fouriertransforms, or in some instances, approximate implementations of thetransforms. For a pixilated image, numerical or other signal processingmethods may give exact implementations through use of the discreteversion of the fractional Fourier transform [10].

Additional computation methods that are possible include one or more of:

-   -   dropping the leading scalar complex-valued phase term (which        typically has little or no effect on the image);    -   decomposing the fractional Fourier transform as a        pre-multiplication by a “phase chirp” e^(icz2), taking a        conventional Fourier transform with appropriately scaled        variables, and multiplying the result by another “phase chirp;”        and    -   changing coordinate systems to Wigner form:

$\begin{matrix}\{ {\frac{( {x + y} )}{w},\frac{( {x - y} )}{w}} \} & (13)\end{matrix}$If desired, any of these just-described computation methods can be usedwith the approximating methods described below.

Other embodiments provide approximation methods for realizing thecorrective fractional Fourier transform operation. For a non-pixilatedimage, numerical or other signal processing methods can giveapproximations through:

-   -   finite-order discrete approximations of the integral        representation;    -   finite-term discrete approximations by means of the Hermite        expansion representation; and    -   the discrete version of the fractional Fourier transform [10].

Classical approximation methods [11, 12] may be used in the latter twocases to accommodate particular engineering, quality, or costconsiderations.

In the case of Hermite expansions, the number of included terms may bedetermined by analyzing the Hermite expansion of the image data, shouldthis be tractable. In general, there will be some value in situationswhere the Hermite function expansion of the image looses amplitude asthe order of the Hermite functions increases. Hermite function orderswith zero or near-zero amplitudes may be neglected entirely from thefractional Fourier computation due to the eigenfunction role of theHermite functions in the fractional Fourier transform operator.

One method for realizing finite-order discrete approximations of theintegral representation would be to employ a localized perturbation orTaylor series expansion of the integral representation. In principal,this approach typically requires some mathematical care in order for theoperator to act as a reflection operator (i.e., inversion of eachhorizontal direction and vertical direction as with the lens law) sincethe kernel behaves as a generalized function (delta function), and hencethe integral representation of the fractional Fourier transform operatorresembles a singular integral.

In a compound lens or other composite optical system, the reflectionoperator may be replaced with the identity operator, which also involvesvirtually identical delta functions and singular integrals as is knownto those familiar in the art. However, this situation is fairly easy tohandle as a first or second-order Taylor series expansion. The requiredfirst, second, and any higher-order derivatives of the fractionalFourier transform integral operator are readily and accurately obtainedsymbolically using available mathematical software programs, such asMathematica or MathLab, with symbolic differential calculuscapabilities. In most cases, the zero-order term in the expansion willbe the simple reflection or identity operator. The resulting expansionmay then be numerically approximated using conventional methods.

Another method for realizing finite-order discrete approximations of theintegral representation would be to employ the infinitesimal generatorof the fractional Fourier transform, that is, the derivative of thefractional Fourier transform with respect to the power of the transform.This is readily computed by differentiating the Hermite functionexpansion of the fractional Fourier transform, and use of the derivativerule for Hermite functions. Depending on the representation used [5, 14,15], the infinitesimal generator may be formed as a linear combinationof the Hamiltonian operator H and the identity operator I; for the formof the integral representation used earlier, this would be:

$\begin{matrix}{\frac{\mathbb{i}\pi}{4}( {H + I} )} & (14)\end{matrix}$where and the identity operator I simply reproduces the originalfunction, and

$\begin{matrix}{H = {\frac{\partial^{2}}{\partial x^{2}} - x^{2}}} & (15)\end{matrix}$The role of the infinitesimal generator, which can be denoted as A, isto represent an operator group in exponential form, a particular exampleis:F ^(α) =e ^(αA)  (16)For small values of A, one can then approximate e^(αA) as I+(αA), sousing the fact [12] from before (repeated here):(F ^(2±ε) [k(x)])⁻¹ =F ⁻² F ^(∓ε) [k(x)]=F ² F ^(∓ε) [k(x)]=F ^(∓ε) F ²[k(x)]=F ^(∓ε) [k(−x)]  (17)one can then approximate F^(ε) as

$\begin{matrix}{F^{ɛ} = {{I + ( {ɛ\; A} )} = {I + {ɛ\frac{\mathbb{i}\pi}{4}( {\frac{\partial^{2}}{\partial_{x}^{2}} - x^{2} + I} )}}}} & (18)\end{matrix}$These operations can be readily applied to images using conventionalimage processing methods. For non-pixilated images, the original sourceimage can be approximated by two-dimensional sampling, and the resultingpixilated image can then be subjected to the discrete version of thefractional Fourier transform [10].

In cases where the discrete version of the fractional Fourier transform[10] is implemented, the transform may be approximated. Pairs ofstandard two-dimensional matrices, one for each dimension of the image,can be used. As with the continuous case, various types of analogousseries approximations, such as those above, can be used.

Alternatively, it is noted that because of the commutative groupproperty of the fractional Fourier transform, the matrix/tensorrepresentations, or in some realizations even the integrals cited abovemay be approximated by pre-computing one or more fixed step sizes andapplying these respectively, iteratively, or in mixed succession to theimage data.

One exemplary embodiment utilizing a pre-computation technique may bewhere the fractional Fourier transform represents pre-computed, positiveand negative values of a small power, for example 0.01. Negative powerdeviations of increasing power can be had by iteratively applying thepre-computed −0.01 power fractional Fourier transform; for example, thepower −0.05 would be realized by applying the pre-computed −0.01 powerfractional Fourier transform five times. In some cases of adaptivesystem realizations, it may be advantageous to discard some of theresulting image data from previous power calculations. This may beaccomplished by backing up to a slightly less negative power by applyingthe +0.01 power fractional Fourier transform to a last stored, resultingimage.

As a second example of this pre-computation method, pre-computedfractional Fourier transform powers obtained from values of the series2^(1/N) and 2^(−1/N) may be stored or otherwise made available, forexample:{F ^(±1/1024) ,F ^(±1/512) ,F ^(±1/256) ,F ^(±1/128) ,F ^(±1/64), . . .}  (19)Then, for example, the power 11/1024 can be realized by operating on theimage data withF ^(1/1024) ,F ^(1/256) ,F ^(1/128)  (20)where the pre-computed operators used are determined by thebinary-decomposition of the power with respect to the smallest powervalue (here, the smallest value is 1/1024 and the binary decompositionof 11/1024 is 1/1024+ 1/256+ 1/128, following from the fact that11=8+2+1).

Such an approach allows, for example, N steps of resolution to beobtained from a maximum of log₂N compositions of log₂N pre-computedvalues. This approach may be used to calculate fractional powers of anylinear matrix or tensor, not just the fractional Fourier transform.

It is noted that any of the aforementioned systems and methods may beadapted for use on portions of an image rather than the entire image.This permits corrections of localized optical aberrations. Incomplicated optical aberration situations, more than one portion of animage may be processed in this manner, with differing correctiveoperations made for each portion of the image.

It is further noted that the systems and methods described herein mayalso be applied to conventional lens-based optical image processingsystems, to systems with other types of elements obeying fractionalFourier optical models, as well as to widely ranging environments suchas integrated optics, optical computing systems, particle beam systems,electron microscopes, radiation accelerators, and astronomicalobservation methods, among others.

Commercial products and services application are widespread. Forexample, the present invention may be incorporated into film processingmachines, desktop photo editing software, photo editing web sites, VCRs,camcorders, desktop video editing systems, video surveillance systems,video conferencing systems, as well as in other types of products andservice facilities. Four exemplary consumer-based applications are nowconsidered.

-   -   1. One particular consumer-based application is in the        correction of camera misfocus in chemical or digital        photography. Here the invention may be used to process the image        optically or digitally, or some combination thereof, to correct        the misfocus effect and create an improved image which is then        used to produce a new chemical photograph or digital image data        file. In this application area, the invention can be        incorporated into film processing machines, desktop photo        editing software, photo editing web sites, and the like.    -   2. Another possible consumer-based application is the correction        of video camcorder misfocus. Camcorder misfocus typically        results from user error, design defects such as a poorly        designed zoom lens, or because an autofocus function is        autoranging on the wrong part of the scene being recorded.        Non-varying misfocus can be corrected for each image with the        same correction parameters. In the case of zoom lens misfocus,        each frame or portion of the video may require differing        correction parameters. In this application area, the invention        can be incorporated into VCRs, camcorders, video editing        systems, video processing machines, desktop video editing        software, and video editing web sites, among others.    -   3. Another commercial application involves the correction of        image misfocus experienced in remote video cameras utilizing        digital signal processing. Particular examples include video        conference cameras or security cameras. In these scenarios, the        video camera focus cannot be adequately or accessibly adjusted,        and the video signal may in fact be compressed.    -   4. Video compression may involve motion compensation operations        that were performed on the unfocused video image. Typical        applications utilizing video compression include, for example,        video conferencing, video mail, and web-based video-on-demand,        to name a few. In these particular types of applications, the        invention may be employed at the video receiver, or at some        pre-processing stage prior to delivering the signal to the video        receiver. If the video compression introduces a limited number        of artifacts, misfocus correction is accomplished as presented        herein. However, if the video compression introduces a higher        number of artifacts, the signal processing involved with the        invention may greatly benefit from working closely with the        video decompression signal processing. One particular        implementation is where misfocus corrections are first applied        to a full video frame image. Then, for some interval of time,        misfocus correction is only applied to the changing regions of        the video image. A specific example may be where large portions        of a misfocused background can be corrected once, and then        reused in those same regions in subsequent video frames.    -   5. The misfocus correction techniques described herein are        directly applicable to electron microscopy systems and        applications. For example, electron microscope optics employ the        wave properties of electrons to create a coherent optics        environment that obeys the Fourier optics structures as coherent        light (see, for example, John C. H. Spence, High-Resolution        Electron Microscopy, third edition, 2003, Chapters 2-4, pp.        15-88). Electron beams found in electron microscopes have the        same geometric, optical physics characteristics generally found        in coherent light, and the same mathematical quadratic phase        structure as indicated in Levi [1] Section 19.2 for coherent        light, which is the basis of the fractional Fourier transform in        optical systems (see, for example, John C. H. Spence        High-Resolution Electron Microscopy, third edition, 2003,        Chapter 3, formula 3.9, pg. 55).        Misfocused Optical Path Phase Reconstruction

Most photographic and electronic image capture, storage, and productiontechnologies are only designed to operate with image amplitudeinformation, regardless as to whether the phase of the light is phasecoherent (as is the case with lasers) or phase noncoherent (as generallyfound in most light sources). In sharply focused images involvingnoncoherent light formed by classical geometric optics, this lack ofphase information is essentially of no consequence in many applications.

In representing the spatial distribution of light, the phase coefficientof the basis functions can be important; as an example, Figure 3.6, p.62 of Digital Image Processing—Concepts, Algorithms, and ScientificApplications, by Bernd Jahne, Springer-Verlag, New York, 1991[20] showsthe effect of loss and modification of basis function phase informationand the resulting distortion in the image. Note in this case the phaseinformation of the light in the original or reproduced image differsfrom the phase information applied to basis functions used forrepresenting the image.

In using fractional powers of the Fourier transform to represent opticaloperations, the fractional Fourier transform reorganizes the spatialdistribution of an image and the phase information as well. Here thebasis functions serve to represent the spatial distribution of light ina physical system and the phase of the complex coefficients multiplyingeach of the basis functions mathematically result from the fractionalFourier transform operation. In the calculation that leads to thefractional Fourier transform representation of a lens, complex-valuedcoefficients arise from the explicit accounting for phase shifts oflight that occurs as it travels through the optical lens (see Goodman[2], pages 77-96, and Levi [1], pages 779-784).

Thus, when correcting misfocused images using fractional powers of theFourier transform, the need may arise for the reconstruction of relativephase information that was lost by photographic and electronic imagecapture, storage, and production technologies that only capture andprocess image amplitude information.

In general, reconstruction of lost phase information has not previouslybeen accomplished with much success, but some embodiments of theinvention leverage specific properties of both the fractional Fouriertransform and an ideal correction condition. More specifically, what isprovided—for each given value of the focus correction parameter—is thecalculation of an associated reconstruction of the relative phaseinformation. Typically, the associated reconstruction will be inaccurateunless the given value of the focus correction parameter is one thatwill indeed correct the focus of the original misfocused image.

This particular aspect of the invention provides for the calculation ofan associated reconstruction of relative phase information by using thealgebraic group property of the fractional Fourier transform to backcalculate the lost relative phase conditions that would have existed, ifthat given specific focus correction setting resulted in a correctlyfocused image. For convergence of human or machine iterations towards anoptimal or near optimal focus correction, the system may also leveragethe continuity of variation of the phase reconstruction as the focuscorrection parameter is varied in the iterations.

To facilitate an understanding of the phase reconstruction aspect of theinvention, it is helpful to briefly summarize the some of the imagemisfocus correction aspects of the invention. This summary will be madewith reference to the various optical set-ups depicted in FIGS. 4-8, andis intended to provide observational details and examples of where andhow the relative phase reconstruction may be calculated (FIG. 9) andapplied (FIG. 10).

Misfocus Correction

FIG. 4 shows a general optical environment involving sources ofradiating light 400, a resulting original light organization(propagation direction, amplitude, and phase) 401 and its constituentphotons. Optical element 402 is shown performing an image-formingoptical operation, causing a modified light organization (propagationdirection, amplitude, and phase) 403 and its constituent photons,ultimately resulting in observed image 404. This figure shows that foreach light organization 401, 403 of light and photons, the propagationdirection, amplitude, and phase may be determined by a variety ofdifferent factors. For example, for a given propagation media,propagation direction, amplitude, and phase may be determined by suchthings as the separation distance between point light source 400 andoptical element 402, the pixel location in a transverse plane parallelto the direction of propagation, and light frequency/wavelength, amongothers.

FIG. 5 is an optical environment similar to that depicted in FIG. 4, butthe FIG. 5 environment includes only a single point light source 500. Inthis Figure, single point light source 500 includes exemplarypropagation rays 501 a, 501 b, 501 c that are presented to opticalelement 502. The optical element is shown imposing an optical operationon these rays, causing them to change direction 503 a, 503 b, 503 c.Each of the rays 503 a, 503 b, 503 c are shown spatially reconverging ata single point in the plane of image formation 504, which is a focusedimage plane.

FIG. 5 also shows directionally modified rays 503 a, 503 b, 503 cspatially diverging at short unfocused image plane 505 and longunfocused image plane 506, which are each transverse to the direction ofpropagation that is associated with images which are not in sharp focus,which will be referred to as nonfocused image planes. Furtherdescription of the optical environment shown in FIG. 5 will be presentedto expand on phase correction, and such description will be laterdiscussed with regard to FIGS. 9-10.

Reference is now made to FIGS. 6-8, which disclose techniques formathematical focus correction and provides a basis for understanding thephase correction aspect of the present invention. For clarity, the term“lens” will be used to refer to optical element 502, but the discussionapplies equally to other types of optical elements such as a system oflenses, graded-index material, and the like.

FIG. 6 provides an example of image information flow in accordance withsome embodiments of the present invention. As depicted in block 600, anoriginal misfocused image is adapted or converted as may be necessaryinto a digital file representation of light amplitude values 601.Examples of original misfocused images include natural or photographicimages. Digital file 601 may include compressed or uncompressed imageformats.

For a monochrome image, the light amplitude values are typicallyrepresented as scalar quantities, while color images typically involvevector quantities such as RBG values, YUV values, and the like. In someinstances, the digital file may have been subjected to file processessuch as compression, decompression, color model transformations, orother data modification processes to be rendered in the form of an arrayof light amplitude values 602. Monochrome images typically only includea single array of scalar values 602 a. In contrast, color images mayrequire one, two, or more additional arrays, such as arrays 602 b and602 c. A CMYB color model is a particular example of a multiple array,color image.

The array, or in some instances, arrays of light amplitude values 602may then be operated on by a fractional power of the Fourier transformoperation 603. This operation mathematically compensates for lensmisfocus causing the focus problems in the original misfocused image600. A result of this operation produces corrected array 604, and incase of a color model, exemplary subarrays 604 a, 604 b, 604 c resultfrom the separate application of the fractional power of the Fouriertransform operation 603 to exemplary subarrays 602 a, 602 b, 602 c. Ifdesired, each of the corrected subarrays 604 a, 604 b, 604 c may beconverted into a digital file representation of the corrected image 605;this digital file could be the same format, similar format, or anentirely different format from that of uncorrected, original digitalfile representation 601.

FIG. 7 shows an optical environment having nonfocused planes. Thisfigure shows that the power of the fractional Fourier transform operatorincreases as the separation distance between optical lens operation 502and image planes 504, 505 increases, up to a distance matching that ofthe lens law. In accordance with some aspects of the invention, and asexplained in Ludwig [5], Goodman [2], pages 77-96, and Levi [1], pages779-784, an exactly focused image corresponds to a fractional Fouriertransform power of exactly two. Furthermore, as previously described,misfocused image plane 505 lies short of the focused image plane 504,and corresponds to a fractional Fourier transform operation with a powerslightly less than two. The deviation in the power of the fractionalFourier transform operation corresponding to short misfocus image plane505 will be denoted (−ε_(s)), where the subscript “S” denotes “short.”Since an exactly focused image at focused image plane 504 corresponds toa fractional Fourier transform power of exactly two, this short misfocusmay be corrected by application of the fractional Fourier transformraised to the power (+ε_(s)), as indicated in block 701.

By mathematical extension, as described in [5], a long misfocused imageplane 506 that lies at a distance further away from optical element 502than does the focused image plane 504 would correspond to a fractionalFourier transform operation with a power slightly greater than two. Thedeviation in the power of the fractional Fourier transform operationcorresponding to long misfocus image plane 506 will be denoted (+ε_(L)),where the subscript “L” denotes “long.” This long misfocus may becorrected by application of the fractional Fourier transform raised tothe power (−ε_(L)), as indicated in block 702.

Relative Phase Information in the Misfocused Optical Path

In terms of geometric optics, misfocus present in short misfocused imageplane 505 and long misfocused image plane 506 generally correspond tonon-convergence of rays traced from point light source 500, throughoptical element 502, resulting in misfocused images planes 505 and 506.For example, FIGS. 5 and 7 show exemplary rays 501 a, 501 b, 501 cdiverging from point light source 500, passing through optical element502, and emerging as redirected rays 503 a, 503 b, 503 c. The redirectedrays are shown converging at a common point in focused image plane 504.However, it is important to note that these redirected rays converge atdiscreetly different points on misfocused image planes 505 and 506.

FIG. 8 is a more detailed view of image planes 504, 505 and 506. In thisfigure, rays 503 a, 503 b, 503 c are shown relative to focused imageplane 504, and misfocused image planes 505 and 506. This figure furthershows the path length differences that lead to phase shifts of thefocused and unfocused planes result from varying angles of incidence,denoted by θ₁ and θ₂. The distances of rays 503 a, 503 b, 503 c fromoptical element 502 are given by the following table:

TABLE 1 Distance to incidence Distance to Distance to incidence withshort misfocused incidence with with long misfocused Ray plane 505 focusplane 504 plane 506 503a δ₁ ^(S) δ₁ ^(F) δ₁ ^(L) 503b δ₀ ^(S) δ₀ ^(F) δ₀^(L) 503c δ₂ ^(S) δ₂ ^(F) δ₂ ^(L)Simple geometry yields the following inequality relationships:δ₀ ^(S)<δ₀ ^(F)<δ₀ ^(L)  (21)δ₁ ^(S)<δ₁ ^(F)<δ₁ ^(L)  (22)δ₂ ^(S)<δ₂ ^(F)<δ₂ ^(L)  (23)For a given wavelength λ, the phase shift ψ created by adistance-of-travel variation δ is given by the following:ψ=2πδ/λ  (24)so the variation in separation distance between the focus image plane504 and the misfocus image planes 505, 506 is seen to introduce phaseshifts along each ray.

Further, for π/2>θ₁>θ₂>0, as is the case shown in FIG. 8, simpletrigonometry gives:δ₀ ^(F)=δ₁ ^(F) sin θ₁  (25)δ₀ ^(F)=δ₂ ^(F) sin θ₂  (26)1>sin θ₁>sin θ₂>0  (27)which in turn yields the inequality relationships:δ₀ ^(S)<δ₂ ^(S)<δ₁ ^(S)  (28)δ₀ ^(F)<δ₂ ^(F)<δ₁ ^(F)  (29)δ₀ ^(L)<δ₂ ^(L)<δ₁ ^(L)  (30)Again, for a given wavelength λ, the phase shift ψ created by adistance-of-travel variation δ is given by the following:ψ=2πδ/λ  (31)so the variation in separation distance between focused image plane 504and the misfocused image planes 505, 506 is seen to introducenon-uniform phase shifts along each ray. Thus the misfocus of theoriginal optical path involved in creating the original image (forexample, 600 in FIG. 6) introduces a non-uniform phase shift across therays of various incident angles, and this phase shift varies with thedistance of separation between the positions of misfocused image planes505, 506, and the focused image plane 504.

Referring again to FIGS. 6 and 7, an example of how a misfocused image600 may be corrected will now be described. A misfocused image requiringcorrection will originate either from short misfocused plane 505 or longmisfocused plane 506. In situations where misfocused image 600originates from short misfocused plane 505, misfocus correction may beobtained by applying a fractional Fourier transform operation raised tothe power (+ε_(s)), as indicated in block 701. On the other hand, insituations where misfocused image 600 originates from long misfocusedplane 506, misfocus correction may be obtained by applying a fractionalFourier transform operation raised to the power (−ε_(L)), as indicatedin block 702.

In general, the fractional Fourier transform operation creates resultsthat are complex-valued. In the case of the discrete fractional Fouriertransform operation, as used herein, this operation may be implementedas, or is equivalent to, a generalized, complex-valued arraymultiplication on the array image of light amplitudes (e.g., φ). In thesignal domain, complex-valued multiplication of a light amplitude arrayelement, ν_(ij), by a complex-valued operator element φ, results in anamplitude scaling corresponding to the polar or phasor amplitude of φ,and a phase shift corresponding to the polar or phasor phase of φ.

FIG. 9 shows a series of formulas that may be used in accordance withthe present invention. As indicated in block 901, the fractional Fouriertransform operation array (FrFT) is symbolically represented as theproduct of an amplitude information array component and a phaseinformation array component. The remaining portions of FIG. 9 illustrateone technique for computing phase correction in conjunction with thecorrection of image misfocus.

For example, block 902 shows one approach for correcting image misfocus,but this particular technique does not provide for phase correction.Image correction may proceed by first noting that the misfocused opticscorresponds to a fractional Fourier transform of power 2-ε for someunknown value of ε; here ε may be positive (short-misfocus) or negative(long-misfocus). Next, the fractional Fourier transform may bemathematically applied with various, systematically selected trialpowers until a suitable trial power is found. A particular example maybe where the trial power is effectively equal to the unknown value of ε.The resulting mathematically corrected image appears in focus and acorrected image is thus produced.

Referring still to FIG. 9, block 903 depicts the misfocus correctiontechnique of block 901, as applied to the approach shown in block 902.This particular technique accounts for the amplitude and phasecomponents of the optical and mathematical fractional Fourier transformoperations. In particular, there is an amplitude and phase for themisfocused optics, which led to the original misfocused image 600.

As previously noted, conventional photographic and electronic imagecapture, storage, and production technologies typically only process oruse image amplitude information, and were phase information is notrequired or desired. In these types of systems, the relative phaseinformation created within the original misfocused optical path is lostsince amplitude information is the only image information that isconveyed. This particular scenario is depicted in block 904, which showsoriginal misfocused image 600 undergoing misfocus correction, eventhought its relative phase information is not available. In allapplicable cases relevant to the present invention (for example, 0<ε<2),the phase information is not uniformly zero phase, and thus the missingphase information gives an inaccurate result (that is, not equal to thefocused case of the Fourier transform raised to the power 2) for whatshould have been the effective correction.

Relative Phase Restoration

In accordance with some embodiments, missing phase information may bereintroduced by absorbing it within the math correction stage, as shownblock 905. This absorbing technique results in a phase-restored mathcorrection of the form:Φ(F ^(2-γ))F ^(γ)  (32)where the following symbolic notation is used:Φ(X)=phase(X)  (33)

In the case where γ is close enough to be effectively equal to ε, thephase correction will effectively be equal to the value necessary torestore the lost relative phase information. Note that this expressiondepends only on γ, and thus phase correction may be obtained bysystematically iterating γ towards the unknown value of ε, which isassociated with the misfocused image. Thus the iteration, computation,manual adjustment, and automatic optimization systems, methods, andstrategies of non-phase applications of image misfocus correction may beapplied in essentially the same fashion as the phase correctingapplications of image misfocus correction by simply substituting F^(γ)with Φ(F^(2-γ))F^(γ) in iterations or manual adjustments.

FIG. 10 provides an example of image information flow in accordance withsome embodiments of the invention. This embodiment is similar to FIG. 6is many respects, but the technique shown in FIG. 10 further includesphase restoration component 1001 coupled with focus correction component603. In operation, image array 602 is passed to phase restorationcomponent 1001, which pre-operates on image array 602. After thepre-operation calculation has been performed, fractional Fouriertransform operation 603 is applied to the image array.

Numerical Calculation of Relative Phase Restoration

Next, the calculation of the phase-restored mathematical correction isconsidered. Leveraging two-group antislayery properties of thefractional Fourier transform operation, the additional computation canbe made relatively small.

In the original eigenfunction/eigenvector series definitions for boththe continuous and discrete forms of the fractional Fourier transform ofpower α, the nth eigenfunction/eigenvectors are multiplied by:e ^(−inπα/2)  (34)Using this equation and replacing α with (2−γ) gives:

$\begin{matrix}\begin{matrix}{{\mathbb{e}}^{{- {\mathbb{i}}}\; n\;{{\pi{({2 - \gamma})}}/2}} = {{\mathbb{e}}^{{- {\mathbb{i}}}\; n\;\pi}{\mathbb{e}}^{{- {\mathbb{i}}}\; n\;{{\pi{({- \gamma})}}/2}}}} \\{= {( {- 1} )^{n}{\mathbb{e}}^{{- {\mathbb{i}}}\; n\;{{\pi{({- \gamma})}}/2}}}}\end{matrix} & (35)\end{matrix}$for both the continuous and discrete forms of the fractional Fouriertransform. Note that the following equation:e ^(−inπ(−γ))  (36)can be rewritten as:e ^(−inπ(−γ)) =e ^(inπγ)=(e ^(−inπγ))*  (37)where (X)* denotes the complex conjugate of X.

Also, because the nth Hermite function h_(n)(y) is odd in y for odd n,and even in y for even n, such that:h _(n)(−y)=(−y)=(−1)^(n) h _(n)(−y)  (38)so that in the series definition the nth term behaves as:

$\begin{matrix}\begin{matrix}{{{h_{n}(x)}{h_{n}(y)}{\mathbb{e}}^{{- {\mathbb{i}}}\; n\;{{\pi{({2 - \gamma})}}/2}}} = {{h_{n}(x)}{h_{n}(y)}( {- 1} )^{n}{\mathbb{e}}^{{- {\mathbb{i}}}\; n\;{{\pi{({- \gamma})}}/2}}}} \\{= {{h_{n}(x)}{h_{n}( {- y} )}{\mathbb{e}}^{{- {\mathbb{i}}}\; n\;{{\pi{({- \gamma})}}/2}}}} \\{= {{h_{n}(x)}{h_{n}( {- y} )}( {\mathbb{e}}^{{- {\mathbb{i}}}\; n\;{\pi\gamma}} )^{*}}}\end{matrix} & (39)\end{matrix}$For both the continuous and discrete forms of the fractional Fouriertransform, replacing h_(n)(y) with h_(n)(−y) is equivalent to reversing,or taking the mirror image, of h_(n)(y). In particular, for the discreteform of the fractional Fourier transform, this amounts to reversing theorder of terms in the eigenvectors coming out of the similaritytransformation, and because of the even-symmetry/odd-antisymmetry of theHermite functions and the fractional Fourier transform discreteeigenvectors, this need only be done for the odd number eigenvectors.

Further, since the Hermite functions and discrete Fourier transformeigenvectors are real-valued, the complex conjugate can be taken on theentire term, not just the exponential, as shown by:h _(n)(x)h _(n)(−y)(e ^(−inπγ))*=[h _(n)(x)h _(n)(−y)e ^(−inπγ)]*  (40)Since complex conjugation commutes with addition, all these series termscan be calculated and summed completely before complex conjugation, andthen one complex conjugation can be applied to the sum, resulting in thesame outcome.

The relative phase-restored mathematical correction can thus becalculated directly, for example, by the following exemplary algorithmor its mathematical or logistic equivalents:

-   -   1. For a given value of γ, compute F^(γ) using the Fourier        transform eigenvectors in an ordered similarity transformation        matrix;    -   2. For the odd-indexed eigenvectors, either reverse the order or        the sign of its terms to get a modified similarity        transformation;    -   3. Compute the complete resulting matrix calculations as would        be done to obtain a fractional Fourier transform, but using this        modified similarity transformation;    -   4. Calculate the complex conjugate of the result of        operation (3) to get the phase restoration, (Φ(F^(γ)))*; and    -   5. Calculate the array product of the operation (1) and        operation (4) to form the phase-restored focus correction        (Φ(F^(γ)))*F^(γ).

As an example of a mathematical or logistic equivalent to the justdescribed series of operations, note the commonality of the calculationsin operations (1) and (3), differing only in how the odd-indexedeigenvectors are handled in the calculation, and in one version, only bya sign change. An example of a mathematical or logistic equivalent tothe above exemplary technique would be:

-   -   1. For a given value of γ, partially compute F^(γ) using only        the even-indexed Fourier transform eigenvectors;    -   2. Next, partially compute the remainder of F^(γ) using only the        odd-indexed Fourier transform eigenvectors;    -   3. Add the results of operation (1) and (2) to get F^(γ)    -   4. Subtract the result of operation (2) from the result of        operation (1) to obtain a portion of the phase restoration;    -   5. Calculate the complex conjugate of the result of        operation (4) to obtain the phase restoration (Φ(F^(γ)))*; and    -   6. Calculate the array product of operations (1) and (4) to form        (Φ(F^(γ)))*F^(γ).

In many situations, partially computing two parts of one similaritytransformation, as described in the second exemplary algorithm, could befar more efficient than performing two full similarity transformationcalculations, as described in the first exemplary algorithm. One skilledin the art will recognize many possible variations with differingadvantages, and that these advantages may also vary with differingcomputational architectures and processor languages.

Embedding Phase Restoration within Image Misfocus Correction

Where relative phase-restoration is required or desired in mathematicalfocus correction using the fractional Fourier transform, phaserestoration element 1001 may be used in combination with focuscorrection element 603, as depicted in FIG. 10.

It is to be realized that in image misfocus correction applicationswhich do not account for phase restoration, pre-computed values of F^(γ)may be stored, fetched, and multiplied as needed or desired. Similarly,in image misfocus correction applications which provide for phaserestoration, pre-computed values of Φ(F^(γ)))*F^(γ) may also be stored,fetched, and multiplied as needed or desired. For example, pre-computedvalues of phase reconstructions may be stored corresponding to powers ofthe fractional Fourier transform, such that the powers are related byroots of the number 2, or realized in correspondence to binaryrepresentations of fractions, or both. In these compositions, care mayneed to be taken since the array multiplications may not freely commutedue to the nonlinear phase extraction steps.

Each of the various techniques for computing the phase-restored focuscorrection may include differing methods for implementing pre-computedphase-restorations. For example, in comparing the first and secondexemplary algorithms, predominated values may be made and stored for anyof:

-   -   First example algorithm operation (5) or its equivalent second        example algorithm operation (6);    -   First example algorithm operation (4) or its equivalent second        example algorithm operation (5); and    -   Second example algorithm operations (1) and (2) with additional        completing computations provided as needed.

Again, it is noted that these phase restoration techniques can apply toany situation involving fractional Fourier transform optics, includingelectron microscopy processes and the global or localized correction ofmisfocus from electron microscopy images lacking phase information.Localized phase-restored misfocus correction using the techniquesdisclosed herein may be particularly useful in three-dimensional,electron microscopy and tomography where a wide field is involved in atleast one dimension of imaging.

It is also noted that the various techniques disclosed herein may beadapted for use on portions of an image rather than the entire image.This permits corrections of localized optical aberrations. Incomplicated optical aberration situations, more than one portion may beprocessed in this manner, in general with differing correctiveoperations made for each portion of the image.

Iterative Fractional Fourier Transform Computation EnvironmentsLeveraging the Structure of the Similarity Transformation

It is also possible to structure computations of the fractional Fouriertransform operating on a sample image array or sampled function vectorso that portions of the computation may be reused in subsequentcomputations. This is demonstrated in the case of a similaritytransformation representation of the fractional Fourier transform inFIG. 11. The general approach applies to vectors, matrices, and tensorsof various dimensions, other types of multiplicative decompositionalrepresentations of the fractional Fourier transform, and other types ofoperators. In particular the approach illustrated in FIG. 11 may bedirectly applied to any diagonalizable linear matrix or tensor, not justthe fractional Fourier transform.

FIG. 11 illustrates the action of a diagonalizable matrix, tensor, orlinear operator on an underlying vector, matrix, tensor, or function1101 to produce a resulting vector, matrix, tensor, or function 1105. Inthis approach, the chosen diagonalizable matrix, tensor, or linearoperator 1100 is decomposed in the standard way into a diagonalizedrepresentation involving the ordered eigenvectors, eigenfunctions, etc.arranged to form a similarity transformation 1102, a diagonal matrix,tensor, or linear operator of eigenvalues 1103 with eigenvalues arrangedin an ordering corresponding to the ordering of the similaritytransformation 1102, and the inverse similarity transformation 1104(equivalent to the inverse of similarity transformation 1102, i.e., theproduct of the inverse similarity transformation 1104 and similaritytransformation 1102 is the identity). As described earlier, arbitrarypowers of the chosen diagonalizable matrix, tensor, or linear operator1100 may be obtained by taking powers of the diagonal matrix, tensor, orlinear operator of eigenvalues 1103, which amounts to simply raisingeach eigenvalue to the desired power.

In particular, to obtain the α power of the chosen diagonalizablematrix, tensor, or linear operator 1100, one simply raises eacheigenvalue in the diagonal matrix, tensor, or linear operator ofeigenvalues 1103 to the α power. This is noted as Λ^(α) in FIGS. 4 and5. Although the discussion thus far is for a very general case, theunderlying vector, matrix, tensor, or function 1101 for the focuscorrection task will be the unfocused image pixel array U, the resultingvector, matrix, tensor, or function 1102 will be the trial “corrected”image pixel array C 1105, the similarity transformation 1102 will be anordered collection of eigenvectors of the discrete Fourier transform oftwo dimensions (a 4-tensor, but simply the outer product of two2-dimensional matrices, each diagonalized in the standard fashion), andthe diagonal matrix, tensor, or linear operator of eigenvalues 1103raised to the power a amounts to the outer product of two 2-dimensionalmatrices, each with eigenvalues of the discrete Fourier transformarranged in corresponding order to that of P, the similaritytransformation 1102.

It is possible to reuse parts of calculations made utilizing thisstructure in various application settings. In a first exemplaryapplication setting, the image is constant throughout but the power αtakes on various values, as in an iteration over values of α in anoptimization loop or in response to a user-adjusted focus control. Inthis first exemplary application setting, the product 1106 of thesimilarity transformation P 1102 and the unfocused image data U 1101 canbe executed once to form a precomputable and reusable result. Then anynumber of α-specific values of the product of the inverse similaritytransformation P⁻¹ 1104 and the diagonal power Λ^(α) 1103 can becomputed separately to form various values of the α-specific portion1107, and each may be multiplied with the precalculated reusable result1106. Further, since diagonal power Λ^(α) 1103 is indeed diagonal,multiplication of it by inverse similarity transformation P⁻¹ 1104amounts to a “column” multiplication of a given column of inversesimilarity transformation P⁻¹ 1104 by the eigenvalue in thecorresponding “column” of the diagonal power Λ^(α) 1103. Thisconsiderably simplifies the required computations in a computationalenvironment iterating over values of α.

FIG. 12 is a flowchart showing exemplary operations for approximatingthe evolution of images propagating through a physical medium, inaccordance with embodiments of the invention. This approximation may beachieved by calculating a fractional power of a numerical operator,which is defined by the physical medium and includes a diagonalizablenumerical linear operator raised to a power (α). In block 1240, aplurality of images are represented using an individual data array foreach of the plurality of images. Block 1245 indicates that the numericaloperator is represented with a linear operator formed by multiplying anordered similarity transformation operator (P) by acorrespondingly-ordered diagonal operator (Λ). The result of this ismultiplied by an approximate inverse (P⁻¹) of the ordered similaritytransformation operator (P).

Next, in block 1250, the diagonal elements of thecorrespondingly-ordered diagonal operator (Λ) are raised to the power(α) to produce a fractional power diagonal operator. Block 1255 includesmultiplying the fractional power diagonal operator by an approximateinverse of the ordered similarity transformation operator (P⁻¹) toproduce a first partial result. Block 1260 includes multiplying a dataarray of one of the plurality of images by the ordered similaritytransformation operator (P) to produce a modified data array. Then,block 1265 includes multiplying the modified data array by the firstpartial result to produce the fractional power of the numericaloperator. If desired, the operations depicted in block 1260 and 1265 maybe repeated for each of the plurality of images.

A second exemplary application is one in which parts of previouscalculations made using the structure of FIG. 11 are reused. In thisembodiment, the power α is constant throughout but the image U 1101 ischanged. This embodiment would pertain to a fixed correction that may berepeatedly applied to a number of image files, for example. Thisembodiment may be used to correct a systemic misfocus episode involvinga number of image files U 1101, or implemented in situations in which aparticular value of the power α is to be applied to a plurality ofregions of a larger image. In this situation, the α-specific portion1107 of the calculation can be executed once to form a precomputable andreusable result. Then any number of image specific calculations 1106 maybe made and multiplied with this α-specific precomputable and reusableresult 1107. Again, since diagonal power Λ^(α) 1103 is indeed diagonal,multiplication of it by inverse similarity transformation P⁻¹ 1104amounts to a “column” multiplication of a given column of inversesimilarity transformation P⁻¹ 1104 by the eigenvalue in thecorresponding “column” of the diagonal power Λ^(α) 1103, considerablysimplifying the required computations in a computational environmentiterating over a plurality of image files, for example.

FIG. 13 is a flowchart showing exemplary operations for approximatingthe evolution of images propagating through a physical medium, inaccordance with alternative embodiments of the invention. Thisapproximation may be achieved by calculating a fractional power of anumerical operator, which is defined by the physical medium and includesa diagonalizable numerical linear operator raised to a power (α) havingany one of a plurality of values.

In block 1300, an image may be represented using a data array. Block1305 indicates that the numerical operator is represented with a linearoperator formed by multiplying an ordered similarity transformationoperator (P) by a correspondingly-ordered diagonal operator (Λ). Theresult of this is multiplied by an approximate inverse (P⁻¹) of theordered similarity transformation operator (P).

Next, in block 1310, the diagonal elements of thecorrespondingly-ordered diagonal operator (Λ) are raised to one of theplurality of values of the power (α) to produce a fractional powerdiagonal operator. Block 1315 includes multiplying the fractional powerdiagonal operator by an approximate inverse of the ordered similaritytransformation operator (P⁻¹) to produce a first partial result. Block1320 includes multiplying the data array of the image by the orderedsimilarity transformation operator (P) to produce a modified data array.Then, block 1325 includes multiplying the modified data array by thefirst partial result to produce the fractional power of the numericaloperator. If desired, the operations depicted in blocks 1310 and 1315may be repeated for each of the plurality of values of the power (α).

FIG. 14 comparatively summarizes the general calculations of each of thetwo described exemplary embodiments in terms of the structure andelements of FIG. 11. Image-specific calculations involving matrix ortensor multiplications can be carried out in an isolated step 1114, ascan α-specific calculations in a separate isolated step 1110. Sincediagonal power Λ^(α) 1103 is diagonal, multiplication of it by inversesimilarity transformation P⁻¹ 1104 amounts to a “column” multiplicationof a given column of inverse similarity transformation P⁻¹ 1104 by theeigenvalue in the corresponding “column” of the diagonal power Λ^(α)1103, considerably simplifying the required computations for theα-specific calculation step 1110. Depending on the situation, eitherstep 1114, 1110 may be made once and reused in calculations 1112involving the matrix or tensor multiplication of the result of sharedpre-computed results (i.e., one of 1114, 1110) and iteration-specificresults (i.e., the other of 1114, 1110).

Matching the Centerings of the Numerical Transform and the Modeled LensAction

The discrete fractional Fourier transform is often described as beingbased on the conventional definition of the classical discrete Fouriertransform matrix. Because the classical discrete Fourier transformmatrix has elements with harmonically-related periodic behavior, thereis a shift invariance as to how the transform is positioned with respectto the frequency-indexed sample space and the time-indexed sample space.The classical discrete Fourier transform matrix typically starts withits first-row, first-column element as a constant, i.e., zero frequency(or in some applications, the lowest-frequency sample point), largely asa matter of convenience since the family of periodic behaviors of theelements and time/frequency sample spaces (i.e., periodic in time viaapplication assumption, period in frequency via aliasing phenomena) areshift invariant.

The periodicity structure of underlying discrete Fourier transform basisfunctions (complex exponentials, or equivalently, sine and cosinefunctions) facilitate this elegant shift invariance in the matter ofarbitrary positioning of the indices defining the classical discreteFourier transform matrix. Thus the discrete classical Fourier transformis defined with its native-zero and frequency-zero at the far edge ofthe native-index range and frequency-index range. An example of this isthe matrix depicted in FIG. 15. In this figure, the left-most column andtop-most row, both having all entries with a value of 1, denote thenative-zero and frequency-zero assignments to the far edge of thenative-index range and frequency-index range (notinge^((j)(0))=e^((0)(k))=e⁰=1).

However, the continuous fractional Fourier transform operates using anentirely different basis function alignment. The continuous fractionalFourier transform is defined with its native-zero and frequency-zero atthe center of the native-variable range and frequency-variable range.This is inherited from the corresponding native-zero and frequency-zerocentering of the continuous classical Fourier transform in thefractionalization process. This situation differs profoundly from thediscrete classical Fourier transform and a discrete fractional Fouriertransform defined from it (which, as described above, is defined withits native-zero and frequency-zero at the far edge of the native-indexrange and frequency-index range).

In particular, with respect to performing image propagation modelingwith a fractionalization of a classical discrete Fourier transform, itis further noted that the native-zero and frequency-zero centering thatcorresponds to the continuous fractional Fourier transform naturallymatches the modeling of the optics of lenses or other quadratic phasemedium. In these optical systems, the phase of the light or particlebeam varies as a function of the distance from the center of the lens.This aspect is illustrated in FIGS. 16A through 16C.

Turning now to FIGS. 16A through 16C, light or high-energy particles areshown radiating spherically from a point in source plane 1600,travelling through space or other medium to a thin lens 1601 where theirdirection is changed and redirected to a point in an image planepositioned according to the lens law for a focused image. Because imageplane 1602 is thus positioned, all light or high-energy particle pathsfrom the same source point that are captured by lens 1601 are bent insuch a way that they all converge at the same point in the positionedimage plane 1602. With respect to center line 1603, which is common tothe center of the lens 1601, the point of convergence is locatedantipodally (modulo magnification factor), with respect the center line,from the location of the emitting point in source plane 1600.

FIG. 16A is a side view of source point 1610, which radiates inexemplary diverging paths 1611 a, 1612 a which are then bent by lens1601 into converging paths 1611 b and 1612 b. Converging paths 1611 band 1612 b are shown reconverging at point 1615 in image plane 1602.Modulo any magnification factor induced by the lens law action,convergence point 1615 is at a location opposite (antipodal), withrespect to center line 1603, to the position of source point 1610.

FIG. 16 b is a side view of second source point 1620, which is locatedat a greater distance from center line 1603 as compared to source point1610 of FIG. 16A. Second source point 1620 is shown radiating inexemplary diverging paths 1621 a and 1622 a, which are then bent by lens1601 into converging paths 1621 b and 1622 b. Converging paths 1621 band 1622 b are shown reconverging at point 1625 in image plane 1602.Notably, point 1625 is located at a greater distance from center line1603, as compared to source point 1610 of FIG. 16A. Modulo anymagnification factor induced by the lens law action, convergence point1625 is at a location opposite (antipodal), with respect to center line1603, to the position of source point 1620.

Similarly, FIG. 16C is a side view of third source point 1630, which islocated at a still further distance from center line 1603. Third sourcepoint 1630 is shown radiating along exemplary diverging paths 1631 a and1632 a when are bent by lens 1601 into converging paths 1631 b and 1632b Converging paths 1631 a and 1632 b are shown reconverging at point1635 in image plane 1602. Note that point 1635 is located at a stillgreater distance from center line 1603. Modulo any magnification factorinduced by the lens law action, the convergence point 1635 is at alocation opposite (antipodal), with respect to center line 1603, to theposition of the source point 1630.

FIG. 16D is a top view of image 1650, which is shown having a variety ofcontours 1651, 1652, 1653, and 1654 of constant radial displacement withrespect to the image center. It is to be understood that any point onany of the identified contours, which is located in a source plane, suchas source plane 1600, will give rise to a point that can only lie withina corresponding contour (modulo magnification) in an image plane, suchas image plane 1602. Thus, the path length of a given path from a pointon one contour in source plane 1600, through lens 1601, to itsreconvergence point in the image plane 1602, is rotationally invariantwith respect to lens center line 1603, but is not translationallyinvariant. Thus induced phase shift of each path will be affected byoffsets or shifts in location between the image center line 1603 and thesource and image centers.

The fractional Fourier transform model, at least in the case of coherentlight or high-energy particle beams, can be thought of as performing thephase accounting as the image evolves through the propagation path.Thus, the classical continuous Fourier transform and itsfractionalization match the modeled optics in sharing the notion of ashared zero origin for all images and lenses, while the classicaldiscrete Fourier transform and its fractionalization do not because ofthe above-described offset in zero origin to the far edge of thetransform index range. It is understood that in the foregoing discussionwith respect to lenses applies equally to almost any type ofquadratic-index media, such as GRIN fiber.

The basis functions used in defining the continuous fractional Fouriertransform are the Hermite functions, which are not periodic despitetheir wiggling behavior—the n^(th) Hermite function has only n zeros(i.e., n axis crossings) and no more. Further, the polynomial amplitudefluctuations of each Hermite function is multiplied by a Gaussianenvelope. These non-periodic functions do not have the shift-invarianceproperties of sines, cosines, and complex exponentials. As a result, andof specific note, the shift rules in the time domain and frequencydomain are far more complicated for the fractional Fourier transformthan for the classical Fourier transform.

Thus, brute-force application of the same fractionalization approachused in the continuous case to the classical discrete Fourier transformmatrix (which does not position time and frequency center at zero asdoes the continuous case), could create undesirable artifacts resultingfrom the non-symmetric definition. It is, in effect, similar to definingthe continuous fractional Fourier transform by the fractionalization ofa “one-sided” continuous classical Fourier transform whose range ofintegration is from zero to positive infinity rather than from negativeinfinity to positive infinity. This can be expected to have differentresults. For example, although it can be shown that pairs of Hermitefunctions which are both of odd order or both of even order are indeedorthogonal on the half-line, pairs of Hermite functions which are one ofodd order and one of even order are not orthogonal on the half-line. Asa result, fractionalization of a “one-sided” continuous classicalFourier transform whose range of integration is from zero to positiveinfinity would not have the full collection of Hermite functions as itsbasis and hence its fractionalization would have different propertiesthan that of the “two-sided” continuous fractional Fourier transform.However, it is the fractionalization of the “two-sided” continuousclassical Fourier transform that matches the optics of lenses and otherquadratic phase medium. The fractionalization of a differently definedtransform could well indeed not match the optics of lenses and otherquadratic phase medium.

Hence, the brute-force application of the same fractionalizationapproach to the classical discrete Fourier transform matrix (which doesnot position time and frequency center at zero), could for someimplementations be expected to create artifacts resulting from anon-symmetric definition. Some studies have reported that the discretefractional Fourier transform defined from just such “brute force” directdiagonallization of the classical discrete Fourier transform reportpathologies and non-expected results. It may, then, in someimplementations, be advantageous—or even essential—to align thezero-origin of the discrete fractional Fourier transform with thezero-origin of the lens action being modeled.

There are a number of ways to define a solution to address this concern.A first class of approaches would be to modify the discrete classicalFourier transform so that its zero-origins are centered with respect tothe center of the image prior to fractionalization. This class ofapproaches would match the discrete transform structure to that of theoptics it is used to model. Another class of approaches would be tomodify the image so that the optics being modeled matches thezero-origins alignment of the classical discrete Fourier transformmatrix, and then proceed with its brute-force fractionalization.Embodiments of the invention provide for either of these approaches, acombination of these approaches, or other approaches which match thezero-origins alignment of a discrete Fourier transform matrix and theimage optics being modeled prior to the fractionalization of thediscrete Fourier transform matrix.

An exemplary embodiment of the first class of approaches would be toshift the classical discrete Fourier transform to a form comprisingsymmetry around the zero-time index and the zero-frequency index beforefractionalization (utilizing the diagonalization and similaritytransformation operations). This results in a “centered” classicalfractional Fourier transform, and its fractionalization would result ina “centered” discrete fractional Fourier transform.

The classical discrete Fourier transform, normalized to be a unitarytransformation, can be represented as an L-by-L matrix whose element inrow p and column q is:

$\begin{matrix}{{D\; F\;{T_{(L)}( {p,q} )}} = {\frac{1}{\sqrt{L}}{\mathbb{e}}^{2\pi\;{{\mathbb{i}}{({p - 1})}}{{({q - 1})}/L}}}} & (41)\end{matrix}$The resulting matrix is depicted in FIG. 15.

The unitary-normalized, classical discrete Fourier transform may besimultaneously shifted in both its original and its frequency indices byk units by simply adding or subtracting the offset variable k for eachof those indices:

$\begin{matrix}{{D\; F\;{T_{({k,L})}( {p,q} )}} = {\frac{1}{\sqrt{L}}{\mathbb{e}}^{2{{\pi\mathbb{i}}{({k + p - 1})}}{{({k + q - 1})}/L}}}} & (42)\end{matrix}$The shifted transform may be centered by setting k to the midpoint ofthe index set {1, 2, 3, . . . L}. This is done by settingk=(L+1)/2.  (43)FIG. 17 shows a resulting matrix for the “centered” normalized classicaldiscrete Fourier transform, in which L is taken as an odd integer. Inthis figure, the matrix is bisected vertically and horizontally by acentral row and column of elements having a value of 1. These valuescorrespond to the zero values of the original and frequency indices. Itis noted that if L is an even integer, these terms of value 1 will notoccur, and in this case there will be no term directly representing azero frequency nor the original image center.

Due to the reflective aliasing of negative frequency components intohigher-index frequency samples, the classical discrete Fourier transformis shifted in such a way towards the indices centers would typically notcompromise redundancy and diminished bandwidth due to symmetry aroundzero as might be expected. As an example, consider an exemplary signalcomprising a unit-amplitude cosine wave of frequency 30 offset by aconstant of ¼:cos(2π·30x)+¼  (44)

Both the classical discrete Fourier transform and the classicalcontinuous Fourier transform naturally respond to the complexexponential representation, specifically[½e ^(i60πx)+½e ^(−i60πx)]+¼  (45)as is well known to those skilled in the art. In the unshifted (i.e.,k=0) case, the classical discrete Fourier transform acts on a signalsuch as this would produce a frequency-domain output like that depictedin FIG. 18A. In this figure, the classical discrete Fourier transform isshown acting on a signal sampled at 201 sample points {0, 1, 2, 3, . . ., 199, 200}. The constant term of ¼ appears at zero frequency (frequencypoint 1 in the sequence), the positive frequency component ½e^(i60πx)term appears at frequency 30 (frequency point 31 in the sequence) andthe negative frequency component ½e^(−i60πx) term is reflectedback—through the far edge of the sampling domain at frequency 200(frequency point 201)—by the aliasing process to a location at frequency170 (frequency point 171 in the sequence).

In contrast, FIG. 18B illustrates the “centered” classical discreteFourier transform acting on the same signal. Here the domain of thesampling and frequency indices range from −100 to +100, specifically{−100, −199, . . . −2, −1, 0, 1, 2, . . . , 99, 100}. The constant termof ¼ appears at zero frequency (frequency point 101 in the sequence),the positive frequency component ½e^(i60πx) term appears at frequency 30(frequency point 131 in the sequence), but here the negative frequencycomponent ½ e^(−i60πx) term appears—without aliasing—at frequency −30(frequency point 71 in the sequence).

This leads to direct interpretations of positive frequency and negativefrequency discrete impulses that correspond with positive frequency andnegative frequency Dirac delta functions that would appear as theclassical continuous Fourier transform. More importantly, thisre-configuring of the computational mathematics of the underlyingdiscrete Fourier transform matrix gives a far more analogousfractionalization to that of the continuous fractional Fourier transformthan the discrete fractional Fourier transform described in mostpublications (which are based on the unshifted classical discreteFourier transform matrix definition). Of course, care must be taken toavoid artifacts created by frequency aliasing effects as would be knownto, clear, and readily addressable to one skilled in thewell-established art of frequency-domain numerical image processing.

For a monochrome rectangular N×M image X(r,s), the unshifted classicaldiscrete Fourier transform result Y(m,n) is, as is well known to oneskilled in the art, given by an expression such as:

$\begin{matrix}{{Y( {n,m} )} = {\sum\limits_{s = 1}^{N}{\sum\limits_{r = 1}^{M}{D\; F\;{T_{(M)}( {s,M} )}D\; F\;{T_{(N)}( {r,n} )}{X( {r,s} )}}}}} & (46)\end{matrix}$where DFT_((L))(p,q) is as given above, though typically normalized by1/L rather than

$\frac{1}{\sqrt{L}}.$Incorporating the shift to centered positions of k_(M)=(M+1)/2 andk_(N)=(N+1)/2, one obtains:

$\begin{matrix}{{Y( {n,m} )} = {\sum\limits_{s = {- \frac{N + 1}{2}}}^{\frac{N + 1}{2}}{\sum\limits_{r = {- \frac{M + 1}{2}}}^{\frac{M + 1}{2}}{D\; F\;{T_{({{{\lbrack{M + 1}\rbrack}/2},M})}( {s,M} )}D\; F\;{T_{({{{\lbrack{N + 1}\rbrack}/2},N})}( {r,n} )}{X( {r,s} )}}}}} & (47)\end{matrix}$Taking the fractional power (a) of each unshifted classical discreteFourier transform matrix as described in previous sections, the overallcomputation will become:

$\begin{matrix}{{Y( {n,m} )} = {\sum\limits_{s = {- \frac{N + 1}{2}}}^{\frac{N + 1}{2}}{\sum\limits_{r = {- \frac{M + 1}{2}}}^{\frac{M + 1}{2}}{D\; F\;{T_{({{{\lbrack{M + 1}\rbrack}/2},M})}^{\alpha}( {s,M} )}D\; F\;{T_{({{{\lbrack{N + 1}\rbrack}/2},N})}^{\alpha}( {r,n} )}{X( {r,s} )}}}}} & (48)\end{matrix}$Finally, adapting the image notation to unfocused source image U(r,s)and corrected image C(m,n) yields the overall operation:

$\begin{matrix}{{C( {n,m} )} = {\sum\limits_{s = {- \frac{N + 1}{2}}}^{\frac{N + 1}{2}}{\sum\limits_{r = {- \frac{M + 1}{2}}}^{\frac{M + 1}{2}}{D\; F\;{T_{({{{\lbrack{M + 1}\rbrack}/2},M})}^{\alpha}( {s,M} )}D\; F\;{T_{({{{\lbrack{N + 1}\rbrack}/2},N})}^{\alpha}( {r,n} )}{U( {r,s} )}}}}} & (49)\end{matrix}$

In the more general case, one can leave the centerings k_(M) and k_(N)unspecified so that they may be set to zero to obtain the unshiftedversion, or they may be set to k_(M)=(M+1)/2 and k_(N)=(N+1)/2 to obtainthe centered version. Embodiments of the invention provide for both ofthese as well as other equivalent or analogous formulations. Of course,it is understood by one skilled in the art that all the summingoperations above may be readily expressed as matrix and tensoroperations, with and within the techniques of fractionalization,iterative computation, reuse of stored and/or precomputed values,handling of color images, etc., may be directly and straightforwardlyapplied.

The contrasting second class of approaches for aligning the center ofthe numerical transform and that of the images involves adapting theimages to the centering of the numerical transforms. An exemplaryembodiment of this second class of approaches could begin with thepartition of an original image into, for example, four quadrant imagesseparated by a pair of perpendicular lines that intersect at the centerof the original image.

Referring now to FIG. 19, image 1900 is shown having height h and widthw. The image has a natural center 1950 which corresponds to that of thecenter line of the lens or equivalent optical element as describedabove. Vertical edges 1912 and 1913 are indexed with extremes of −h/2and +h/2 with center index zero as with example element 1901. Similarly,horizontal edges 1902 and 1903 are indexed with extremes of −w/2 and+w/2 with center index zero as with example element 1911. In contrast tothe first class of approaches described above, this exemplary embodimentof a second class of approaches utilizes an image which is separatedalong perpendicular lines into four parts, as illustrated in FIG. 20.

Each of the four distinct quadrant parts 2001, 2002, 2003, and 2004 ofthe original image 1900 may be treated as an isolated image 2001 a, 2002a, 2003 a, and 2004 a with its own zero-origins in a far corner,matching in abstraction the zero-corner attribute of the discreteclassical Fourier transform. The coordinates of the four quadrant imagesmay then be interpreted or realigned, as denoted by blocks 2001 b, 2002b, 2003 b, and 2004 b to match the coordinate system of the discreteclassical Fourier transform.

The brute-force fractionalization of the discrete classical Fouriertransform can be applied to each of these to obtain four quadranttransformed images, denoted by transformed images 2001 c, 2002 c, 2003c, and 2004 c. The transformed images 2001 c, 2002 c, 2003 c, and 2004 ccan then be interpretively or operationally realigned and reassembled,resulting in reassembled images 2001 d, 2002 d, 2003 d, and 2004 d. Thereassembled images 2001 d, 2002 d, 2003 d, and 2004 d may then be usedto form the larger composite image 2010 that matches the quadrantconfiguration of the original image 1900. Without undergoing anyadditional processing, composite image 2010 would typically have edgeeffects at the quadrant boundaries.

If desired, these edge effects may be resolved, softened, or eliminatedby performing additional calculations. For example, a second pair, ormore, of perpendicular lines can be used to partition the original imagein a manner that differs from that which is shown in FIG. 20 (forexample, rotated and/or shifted with respect the original pair). Then,the process shown and described in conjunction with FIG. 20 may then beapplied to these distinctly different quadrants as well. The generatedcalculations may be cross-faded or pre-emphasized and added to produce acomposite image with significantly diminished boundary edge-effectartifacts.

Software and Hardware Realizations

Typically, each of the various techniques described herein are invariantof which underlying discrete Fourier transform matrix is fractionalizedto define the discrete fractional Fourier transform matrix. Althoughembodiments of the present invention may be implemented using theexemplary series of operations depicted in the figures, those ofordinary skill in the art will realize that additional or feweroperations may be performed. Moreover, it is to be understood that theorder of operations shown in these figures is merely exemplary and thatno single order of operation is required. In addition, the variousprocedures and operations described herein may be implemented in acomputer-readable medium using, for example, computer software,hardware, or some combination thereof.

For a hardware implementation, the embodiments described herein may beimplemented within one or more application specific integrated circuits(ASICs), digital signal processors (DSPs), digital signal processingdevices (DSPDs), programmable logic devices (PLDs), field programmablegate arrays (FPGAs), processors, controllers, micro-controllers,microprocessors, other electronic units designed to perform thefunctions described herein, or a combination thereof.

For a software implementation, the embodiments described herein may beimplemented with modules, such as procedures, functions, and the like,that perform the functions and operations described herein. The softwarecodes can be implemented with a software application written in anysuitable programming language and may be stored in a memory unit, suchas memory 208 or memory 306, and executed by a processor. The memoryunit may be implemented within the processor or external to theprocessor, in which case it can be communicatively coupled to theprocessor using known communication techniques. Memory 306 may beimplemented using any type (or combination) of suitable volatile andnon-volatile memory or storage devices including random access memory(RAM), static random access memory (SRAM), electrically erasableprogrammable read-only memory (EEPROM), erasable programmable read-onlymemory (EPROM), programmable read-only memory (PROM), read-only memory(ROM), magnetic memory, flash memory, magnetic or optical disk, or othersimilar memory or data storage device.

The programming language chosen should be compatible with the computingplatform according to which the software application is executed.Examples of suitable programming languages include C and C++. Theprocessor may be a specific or general purpose computer such as apersonal computer having an operating system such as DOS, Windows, OS/2or Linux; Macintosh computers; computers having JAVA OS as the operatingsystem; graphical workstations such as the computers of Sun Microsystemsand Silicon Graphics, and other computers having some version of theUNIX operating system such as AIX or SOLARIS of Sun Microsystems; or anyother known and available operating system, or any device including, butnot limited to, laptops and hand-held computers.

While the invention has been described in detail with reference todisclosed embodiments, various modifications within the scope of theinvention will be apparent to those of ordinary skill in thistechnological field. It is to be appreciated that features describedwith respect to one embodiment typically may be applied to otherembodiments. Therefore, the invention properly is to be construed withreference to the claims.

I claim:
 1. A method for numerically correcting misfocus in originalimage data employing numerical processing, the method comprising:receiving original image data from an external source, wherein at leasta portion of the original image data comprises misfocus, the originalimage data comprising a plurality of spatially-indexed amplitude values,and further comprising a center located relative to the plurality ofspatially-indexed amplitude values; operating on the original image datawith a centered two-dimensional discrete fractional Fourier transform toproduce modified image data, the centered two-dimensional fractionalFourier transform operation configured with a fractional powerparameter, the fractional power parameter set to an initial value;comparing the misfocus of the modified image data with the misfocus ofthe original image data; if necessary, adjusting the fractional powerparameter of the centered two-dimensional fractional Fourier transformoperation to further change the misfocus of the modified image data, andcomparing the misfocus of the modified image data with the misfocus ofthe original image data; wherein applying the Fourier transformoperation to the provided image data is used to generate modified imagedata comprising an improvement in the misfocus of the original imagedata; wherein the centered two-dimensional discrete fractional Fouriertransform is a numerical operator comprising an index-shifted discretefractional Fourier transform operation on the original image data, thenumerical operator comprising original-domain indices andtransform-domain indices, wherein the original-domain indices comprise azero original-domain origin that is centered within the original-domainindices, and the transform-domain indices comprise a zerotransform-domain origin that is centered within the transform-domainindices; and wherein the zero original-domain origin is aligned to thecenter of the original image data and the zero transform-domain originis aligned to the center of the modified image data.
 2. The method ofclaim 1, wherein the misfocus in the original image data was created byimage formation employing a single lens.
 3. The method of claim 1,wherein the misfocus in the original image data was created by imageformation employing system of lenses.
 4. The method of claim 1, whereinthe misfocus in the original image data was created by a image formationemploying graded-index material.
 5. The method of claim 4, wherein themisfocus in the original image data was created by an electronmicroscope.
 6. The method of claim 1, wherein the original image datacomprises an image formed by light.
 7. The method of claim 1, whereinthe original image data comprises an image formed by a particle beam. 8.The method of claim 1, wherein the modified image data is structured topermit correction of misfocus in a selected portion of the originalimage data.
 9. The method of claim 1, wherein the original image datacomprises still-image data.
 10. The method of claim 1, wherein theoriginal image data is an image comprises moving-image data.
 11. Themethod of claim 1, wherein the fractional power parameter is adjustedmanually.
 12. The method of claim 1, wherein the fractional powerparameter is adjusted at least in part manually.
 13. The method of claim1, wherein the fractional power parameter is adjusted automatically. 14.The method of claim 1, wherein the fractional power parameter isadjusted at least in part automatically.
 15. The method of claim 1,wherein the fractional power parameter is adjusted using an iterativeprocess.
 16. The method of claim 1, wherein a measure of misfocus isprovided by a spatial image high-pass filter.
 17. The method of claim 1,wherein a measure of misfocus is provided by a spatial imagedifferentiator.
 18. The method of claim 1, wherein further the originalimage data is obtained via a website interface.
 19. The method of claim1, the method further comprising a website interface for controlling thefractional power parameter.
 20. The method of claim 1, the methodfurther comprising a website interface for transferring the modifiedimage data.